This invention relates generally to processing within a computing environment, and more particularly to computing systems having a multilevel cache hierarchy.
In computers, a cache is a component that improves performance by transparently storing data such that future requests for that data can be served faster. The data that is stored within a cache might be values that have been computed earlier or duplicates of original values that are stored elsewhere (e.g. main memory). If requested data is contained in the cache (cache hit), this request can be served by simply reading the cache, which is comparably faster. Otherwise (cache miss), the data has to be recomputed or fetched from its original storage location, which is comparably slower.
The operation of cache operations in a shared cache are controlled by a cache controller. The cache controller may include a shared pipeline that processes individual requests. A pipeline may be considered as a set of data processing elements connected in series, so that the output of one element is the input of the next one. An instruction pipeline may be used in a computing device to increase instruction throughput (the number of instructions that can be executed in a unit of time). The fundamental idea is to split the processing of a computer instruction into a series of independent steps, with storage at the end of each step. This allows the computer's control circuitry to issue instructions at the processing rate of the slowest step, which is much faster than the time needed to perform all steps at once. The term pipeline refers to the fact that each step is carrying data at once (like water), and each step is connected to the next (like the links of a pipe.)
Every new generation of high performance computer systems bring with them a substantial increase in overall system capacity. This capacity increase brings with it the need for additional resources to process commands through the cache controller. Every command processed in the cache controller requires the use of one or more command queues. These queues are typically task specific, capable of handling only a limited subset of the possible commands processed by the cache controller. The cache controller therefore has many varieties of queues, each dedicated to processing a subset of the commands sent to the cache.
Physical constraints limit the total number of command queues that may exist in the cache controller. This limits the number of each type of command queues in the cache controller. Bursts of commands that require the use of the same queues will end up stalling due to resource availability while large numbers of other queues remain idle. This slows down the processing of these bursts of commands and is an inefficient utilization of resources.